Evaluating, Understanding, and Improving Behavioral Game Theory Models For Predicting Human Behavior in Unrepeated Normal-Form Games

It is common to assume that agents will adopt Nash equilibrium strategies; however, experimental studies have demonstrated that Nash equilibrium is often a poor description of human players’ behavior in unrepeated normal-form games. In this paper, we analyze four widely studied models (QRE, Lk, Cognitive Hierarchy, QLk) that aim to describe actual, rather than idealized, human behavior. We performed a meta-analysis of these models, leveraging nine different data sets from the literature, predominantly of two-player games. We begin by evaluating the models’ generalization or predictive performance, asking how well a model fits unseen “test data” after having had its parameters calibrated based on separate “training data”. Surprisingly, we found that the QLk model of Stahl and Wilson (1994) consistently achieved the best performance. Motivated by this finding, we describe methods for analyzing the posterior distributions over a model’s parameters. We found that QLk’s parameters were being set to values that were not consistent with their intended economic interpretation. We thus explored variations of QLk, ultimately identifying a new model family that has fewer parameters, gives rise to more parsimonious parameter values, and achieves better predictive performance.